-
Notifications
You must be signed in to change notification settings - Fork 59
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Multi cell database/mq adoption #746
base: main
Are you sure you want to change the base?
Multi cell database/mq adoption #746
Conversation
3323bbf
to
a1207eb
Compare
Build failed (check pipeline). Post https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/ff2fa0079d2d4d0f9d107db215021a87 ✔️ noop SUCCESS in 0s |
docs_user/modules/proc_migrating-databases-to-mariadb-instances.adoc
Outdated
Show resolved
Hide resolved
0ba8ce7
to
cd5b8f9
Compare
Build failed (check pipeline). Post https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/2bf5c26d9ce042488b038c87505a0f82 ✔️ noop SUCCESS in 0s |
c392e11
to
006e247
Compare
Build failed (check pipeline). Post https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/ef997d9bd22a422fa0a81831c10ce76f ✔️ noop SUCCESS in 0s |
006e247
to
55aec9f
Compare
Build failed (check pipeline). Post https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/221b8eaa480445db948ba454fc1dc6d9 ✔️ noop SUCCESS in 0s |
recheck |
55aec9f
to
ee3defd
Compare
This change depends on a change that failed to merge. Change https://review.rdoproject.org/r/c/rdo-jobs/+/53192 is needed. |
ee3defd
to
03ce37c
Compare
Build failed (check pipeline). Post https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/28720608befb44c49d0f76f95466684b ✔️ noop SUCCESS in 0s |
505e4e8
to
8d81b48
Compare
tests/roles/backend_services/templates/openstack_control_plane.j2
Outdated
Show resolved
Hide resolved
tests/roles/backend_services/templates/openstack_control_plane.j2
Outdated
Show resolved
Hide resolved
MARIADB_IMAGE=registry.redhat.io/rhosp-dev-preview/openstack-mariadb-rhel9:18.0 | ||
endif::[] | ||
SOURCE_MARIADB_IP=172.17.0.2 | ||
$ PASSWORD_FILE="$HOME/overcloud-passwords.yaml" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
see how we match this procedure's commands in docs with those in tests https://github.com/openstack-k8s-operators/data-plane-adoption/pull/746/files#diff-50c1d11b8d50eb791a57420a1cb81ce7bce10dee4ec137c139146ade1ca9dc57R27
8d81b48
to
cdff544
Compare
Build failed (check pipeline). Post https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/513df93417f943f8bc8b6d3849192150 ✔️ noop SUCCESS in 0s |
Build failed (check pipeline). Post https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/95742f210b3246b5b365a50690736dd1 ✔️ noop SUCCESS in 0s |
c3e2073
to
12be61e
Compare
Build failed (check pipeline). Post https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/9341eefb56e64f019fdb05a431eb7bc1 ✔️ noop SUCCESS in 0s |
12be61e
to
cde2221
Compare
This change depends on a change that failed to merge. Change https://review.rdoproject.org/r/c/rdo-jobs/+/53192 is needed. |
Build failed (check pipeline). Post https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/448f64218a864e30bc1e45f7153c8f04 ✔️ noop SUCCESS in 0s |
Provide a static multi-cell config for databases and messaging for adoption guide and tests, which comprises a 3 cells. Keep renaming 'default' cell consistent for single and multi cells: Default becomes cellX (or it can be imported as is, for a multi-cell case only) cell1 becomes mapped to openstack-cell1 osdp node set cell2 becomes mapped to openstack-cell2 osdp node set, etc. cellX (X=3 here) becomes mapped to openstack-cell3. Alternatively, default cell retains its name for the openstack-default osdpns mapping Evaluate podified MariaDB passwords for cells from osp-secret to align the tests with documented commands. Remove no longer needed podified DB password variable. Make ansible and shell variables compute cells aware. Rework vars and secrets YAML values for the source and edpm nodes to not confuse its different naming schemes for cells in OSP/TripleO and RHOSO. Remove cached fact for pulled OSP configuration as it can no longer be generated in a multi-cell setup, where related shell variables become bash arrays. Simplify ENV headers management by collecting in a single place. Adjust storage/storageRequests values to make it better fitting a multi-cell test scenarios. Also provide values in docs and add a comment to adjust them as needed. Remove source_db_root_password as it is directly evaluated from tripleo passwords into an env var. Run mysql commands in individual pods. Finished pods take time to terminate, avoid errors where consequent mysql commands failing because the old and new pod use the same name. Rename nodesets to openstack-cell1, which is needed for adoption of remaining multi-cell aware services in a follow up. Signed-off-by: Bohdan Dobrelia <[email protected]> Fix Signed-off-by: Bohdan Dobrelia <[email protected]>
Declare RUN_OVERRIDES before it is used. Use env vars instead of docs generation conditions to reuse the same code in tests: * Add MARIADB_RUN_OVERRIDES to cover all overrides and client annotations * Add missing definitions for rhoso/ospd namespace specific vars * Use env TRIPLEO_PASSWORDS for all cases as OSPDo still deploys tripleo * Define and use NAMESPACE (default openstack) instead of RHOSO18_NAMESPACE or OSPDO_NAMESPACE. Remove unused rhoso18 ns value (only in these guide). Signed-off-by: Bohdan Dobrelia <[email protected]>
Illustrate how commands in scripts could have comments that become (almost as is) native ascii docs foot-notes. When copying code into docs, the minimal adjustments will be needed, like adding '$' prefix (or '>' for multiline commands). Signed-off-by: Bohdan Dobrelia <[email protected]>
Those will be added back in a follow up, which completes the guide and tests for extra cell2 and cell3. Signed-off-by: Bohdan Dobrelia <[email protected]>
Assume a single cell1 yet. Remove edpm_computes and computes env var from tests as it is not multi-cell aware, and should be no longer needed. The docs still use that env var, it will be removed in multi-cell adoption follow up, where we also cover EDPM multi-cell adoption. This is required as rhe rdo-jobs dependency introduces that change for edpm_nodes and provides a common base for this and future multi-cell follow ups. Signed-off-by: Bohdan Dobrelia <[email protected]>
Signed-off-by: Bohdan Dobrelia <[email protected]>
Signed-off-by: Bohdan Dobrelia <[email protected]>
Co-authored-by: Katie Gilligan <[email protected]>
Use 'mysql_upgrade --skip-version-check' when truing to fix mysqlcheck failure. Signed-off-by: Bohdan Dobrelia <[email protected]>
8c9f19a
to
5fad9f3
Compare
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
This change depends on a change that failed to merge. Change https://review.rdoproject.org/r/c/rdo-jobs/+/53192 is needed. |
testproject and tempest has passed for the 1st time https://review.rdoproject.org/r/c/testproject/+/55584 @jistr I retriggered testproject to include remaining jobs also |
@klgill please approve these docs changes made as you suggested, then we can address other changes in follow up. |
recheck |
Build failed (check pipeline). Post https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/5520cb8282f54f72890da27cfb8bd248 ✔️ noop SUCCESS in 0s |
recheck https://review.rdoproject.org/r/c/testproject/+/55584 \o/ all green! please merge all in the dep chain |
let's please re-iterate in follow up, the technical part of this is done, including testing
This change depends on a change that failed to merge. Change https://review.rdoproject.org/r/c/rdo-jobs/+/53192 is needed. |
recheck |
I looked into the test result there and I see only cell0 and cell1 there on the 18.0 side with all 3 computes mapped to cell1. Is this the expected result? |
Build failed (check pipeline). Post https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/8817c2d033f3422ab613c473a0b984dd ✔️ noop SUCCESS in 0s |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@bogdando I added some comments.
Without the doc build, it's difficult for me to tell how these changes affect the formatting of the doc. When this PR is eventually merged, I will likely need to check the formatting and open a new doc PR.
@@ -396,11 +375,22 @@ endif::[] | |||
+ | |||
<1> Select an existing storage class in your {OpenShiftShort} cluster. | |||
|
|||
This example provides the required infrastructure database and messaging services for 1 Compute cell | |||
named `cell1`. Adjust the names, counts, IP addresses, and numbers, such as `replicas`, `storage`, or `storageRequest`, as needed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
named `cell1`. Adjust the names, counts, IP addresses, and numbers, such as `replicas`, `storage`, or `storageRequest`, as needed. | |
named `cell1`. Adjust the names, counts, IP addresses, and numbers, such as for `replicas`, `storage`, or `storageRequest`, as needed. |
---- | ||
Later on in this guide, the given cells names are referred to by using the environment variable `RENAMED_CELLS`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
---- | |
Later on in this guide, the given cells names are referred to by using the environment variable `RENAMED_CELLS`. | |
---- | |
+ | |
The given cells names are later referred to by using the environment variable `RENAMED_CELLS`. |
<1> `CELLS` and `RENAMED_CELLS` represent changes that are going to be made | ||
after importing databases: the `default` cell takes a new name from `DEFAULT_CELL_NAME`. | ||
In a multi-cell adoption scenario, it might retain its original 'default' name as well. | ||
<2> The `CHARACTER_SET` variable and collation should match the source database. If they do not match, then break foreign key relationships break for any tables that are created in the future as part of database sync. | ||
<3> Complete `SOURCE_MARIADB_IP[*]= ...` data for each cell defined in `CELLS`. Provide records for the cell names and VIP addresses of MariaDB Galera clusters. | ||
<4> For each cell defined in `CELLS`, complete `SOURCE_GALERA_MEMBERS_CELL<X>` with the names of MariaDB Galera cluster members and its IP address. Replace `["standalone.localdomain"]="172.17.0.100"` to real hosts data. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
<1> `CELLS` and `RENAMED_CELLS` represent changes that are going to be made | |
after importing databases: the `default` cell takes a new name from `DEFAULT_CELL_NAME`. | |
In a multi-cell adoption scenario, it might retain its original 'default' name as well. | |
<2> The `CHARACTER_SET` variable and collation should match the source database. If they do not match, then break foreign key relationships break for any tables that are created in the future as part of database sync. | |
<3> Complete `SOURCE_MARIADB_IP[*]= ...` data for each cell defined in `CELLS`. Provide records for the cell names and VIP addresses of MariaDB Galera clusters. | |
<4> For each cell defined in `CELLS`, complete `SOURCE_GALERA_MEMBERS_CELL<X>` with the names of MariaDB Galera cluster members and its IP address. Replace `["standalone.localdomain"]="172.17.0.100"` to real hosts data. | |
<1> `CELLS` and `RENAMED_CELLS` represent changes that are going to be made after you import the databases. The `default` cell takes a new name from `DEFAULT_CELL_NAME`. | |
In a multi-cell adoption scenario, `default` cell might retain its original 'default' name as well. | |
<2> The `CHARACTER_SET` variable and collation should match the source database. If they do not match, then foreign key relationships break for any tables that are created in the future as part of database sync. | |
<3> Add data in `SOURCE_MARIADB_IP[*]= ...` for each cell that is defined in `CELLS`. Provide records for the cell names and VIP addresses of MariaDB Galera clusters. | |
<4> For each cell defined in `CELLS`, in `SOURCE_GALERA_MEMBERS_CELL<X>`, add the names of the MariaDB Galera cluster members and its IP address. Replace `["standalone.localdomain"]="172.17.0.100"` with the real hosts data. |
<4> For each cell defined in `CELLS`, complete `SOURCE_GALERA_MEMBERS_CELL<X>` with the names of MariaDB Galera cluster members and its IP address. Replace `["standalone.localdomain"]="172.17.0.100"` to real hosts data. | ||
|
||
[NOTE] | ||
A standalone {OpenStackPreviousInstaller} environment only creates a 'default' cell, which should be the only `CELLS` value for such a case. The `DEFAULT_CELL_NAME` value should be `cell1`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A standalone {OpenStackPreviousInstaller} environment only creates a 'default' cell, which should be the only `CELLS` value for such a case. The `DEFAULT_CELL_NAME` value should be `cell1`. | |
A standalone {OpenStackPreviousInstaller} environment only creates a 'default' cell, which should be the only `CELLS` value in this case. The `DEFAULT_CELL_NAME` value should be `cell1`. |
The `super` is not a cell, but the top-scope Nova API "upcall" database instance. A super conductor connects to that database. In this guide, the upcall and cells databases are going to use the same password that is | ||
defined in `osp-secret`. Old passwords are only needed to prepare the data exports. | ||
|
||
To get the values for `SOURCE_MARIADB_IP`, query the puppet-generated configurations in a Controller and CellController nodes: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The `super` is not a cell, but the top-scope Nova API "upcall" database instance. A super conductor connects to that database. In this guide, the upcall and cells databases are going to use the same password that is | |
defined in `osp-secret`. Old passwords are only needed to prepare the data exports. | |
To get the values for `SOURCE_MARIADB_IP`, query the puppet-generated configurations in a Controller and CellController nodes: | |
The `super` is the top-scope Nova API upcall database instance. A super conductor connects to that database. In subsequent examples, the upcall and cells databases use the same password that is defined in `osp-secret`. Old passwords are only needed to prepare the data exports. | |
To get the values for `SOURCE_MARIADB_IP`, query the puppet-generated configurations in the Controller and CellController nodes: |
+ | ||
[NOTE] | ||
The `nova`, `nova_api`, and `nova_cell0` databases are included in the same database host. | ||
The `nova`, `nova_api`, and `nova_cell0` databases are inlcuded in the same database host for the main overcloud Heat stack. | ||
ifeval::["{OpenStackPreviousInstaller}" != "director_operator"] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ifeval::["{OpenStackPreviousInstaller}" != "director_operator"] | |
ifeval::["{build_variant}" == "ospdo"] |
+ | ||
[NOTE] | ||
==== | ||
This configuration is required later, during the data plane adoption post-checks. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does "this configuration" refer to the configuration for the neutron-sriov-nic-agent
agents?
---- | ||
+ | ||
** Replace `["standalone.localdomain"]="192.168.122.100"` with the name and IP address of the Compute node. | ||
<1> Complete `CONTROLLER<X>_SSH` settings with SSH connection details for all Controllers, including cell Controllers, of the source {OpenStackPreviousInstaller} cloud. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
<1> Complete `CONTROLLER<X>_SSH` settings with SSH connection details for all Controllers, including cell Controllers, of the source {OpenStackPreviousInstaller} cloud. | |
<1> In the `CONTROLLER<X>_SSH` settings, provide SSH connection details for all Controller nodes, including cell Controller nodes, of the source {OpenStackPreviousInstaller} cloud. |
CONTROLLER3_SSH="ssh -i *<path to SSH key>* root@*<controller-3 IP>*" | ||
endif::[] | ||
---- | ||
* Specify the IP addresses of all Controllers, for example: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
* Specify the IP addresses of all Controllers, for example: | |
* Specify the IP addresses of all Controller nodes, for example: |
---- | ||
ifeval::["{build}" == "downstream"] | ||
<1> Replace `<path_to_SSH_key>` with the path to your SSH key. | ||
<2> Replace `<controller-<X> IP>` with IP addresses of all Controllers. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
<2> Replace `<controller-<X> IP>` with IP addresses of all Controllers. | |
<2> Replace `<controller-<X> IP>` with IP addresses of all Controller nodes. |
Declare RUN_OVERRIDES before it is used.
Use env vars instead of docs generation conditions to reuse the same
code in tests:
tripleo
RHOSO18_NAMESPACE or OSPDO_NAMESPACE. Remove unused rhoso18 ns value
(only in these guide).
Illustrate how commands in scripts could have comments
that become (almost as is) native ascii docs foot-notes.
When copying code into docs, the minimal adjustments will
be needed, like adding '$' prefix (or '>' for multiline commands).
Provide a static multi-cell config for databases and messaging
for adoption guide and tests, which comprises a 3 cells.
Default becomes cellX (or it can be imported as is, for a multi-cell
case only)
cell1 becomes mapped to openstack-cell1 osdp node set
cell2 becomes mapped to openstack-cell2 osdp node set, etc.
cellX (X=3 here) becomes mapped to openstack-cell3. Alternatively,
default cell retains its name for the openstack-default osdpns
mapping
Evaluate podified MariaDB passwords for cells from osp-secret
to align the tests with documented commands. Remove no longer
needed podified DB password variable.
Make ansible and shell variables compute cells aware.
Rework vars and secrets YAML values for the source and edpm
nodes to not confuse its different naming schemes for cells
in OSP/TripleO and RHOSO.
Remove cached fact for pulled OSP configuration as it can no longer
be generated in a multi-cell setup, where related shell variables
become bash arrays.
Simplify ENV headers management by collecting in a single place.
Adjust storage/storageRequests values to make it better fitting
a multi-cell test scenarios. Also provide values in docs and
add a comment to adjust them as needed.
Remove source_db_root_password as it is directly evaluated from
tripleo passwords into an env var.
Run mysql commands in individual pods.
Finished pods take time to terminate, avoid errors where
consequent mysql commands failing because the old and new pod use the
same name.
Rename nodesets to openstack-cell1, which is needed for adoption of
remaining multi-cell aware services in a follow up.
Make edpm_nodes input multi-cell aware.
Assume a single cell1 yet.
Remove edpm_computes and computes env var
from tests as it is not multi-cell aware, and should be no longer
needed. The docs still use that env var, it will be removed in
multi-cell adoption follow up, where we also cover EDPM multi-cell
adoption.
This is required as rhe rdo-jobs dependency introduces that
change for edpm_nodes and provides a common base for this and future
multi-cell follow ups.
Closes: #184
Depends-On: https://review.rdoproject.org/r/c/rdo-jobs/+/53192
Jira: #OSPRH-6548